Papers with data generation pipeline
SCoPE VLM: Selective Context Processing for Efficient Document Navigation in Vision-Language Models (2026.eacl-long)
Copied to clipboard
| Challenge: | Existing methods for document understanding are memory-intensive and impractical for local deployments. |
| Approach: | They propose a document navigation expert that leverages a Chain of Scroll mechanism to selectively and recursively navigate documents, focusing exclusively on relevant segments. |
| Outcome: | The proposed method reduces memory usage and effectively models human-like reading behaviors. |
LEGENT: Open Platform for Embodied Agents (2024.acl-demos)
Copied to clipboard
Zhili Cheng, Zhitong Wang, Jinyi Hu, Shengding Hu, An Liu, Yuge Tu, Pengkai Li, Lei Shi, Zhiyuan Liu, Maosong Sun
| Challenge: | Existing integrations of large language models and large multimodal models are limited . Existing platforms for developing embodied agents are limited and limited based on open-source software. |
| Approach: | They propose an open platform for developing embodied agents using LLMs and LMMs. |
| Outcome: | The proposed platform surpasses GPT-4V in embodied tasks with its model training on LEGENT data. |
Auto-Cypher: Improving LLMs on Cypher generation via LLM-supervised generation-verification framework (2025.naacl-short)
Copied to clipboard
| Challenge: | Graph databases like Neo4j are gaining popularity for handling complex, interconnected data, over traditional relational databases. |
| Approach: | They propose an automated pipeline to generate Cypher queries for Neo4j using LLM-As-Database-Filler, a novel strategy for ensuring Cyphere query correctness. |
| Outcome: | The proposed pipeline generates high quality Cypher data containing 29.8k instances across various domains and queries with varying complexities. |
StressTest: Can YOUR Speech LM Handle the Stress? (2026.findings-acl)
Copied to clipboard
| Challenge: | Recent speech-aware language models (SLMs) have enabled direct audio processing, allowing models to access the full expressive range of spoken language. |
| Approach: | They propose a data generation pipeline that simulates change of meaning implied by stress variation and propose 'stresstest' to evaluate models' ability to distinguish between meanings of speech based on stress pattern. |
| Outcome: | The proposed model outperforms existing models on sentence stress reasoning and detection. |
A Functionality-Grounded Benchmark for Evaluating Web Agents in E-commerce Domains (2026.acl-long)
Copied to clipboard
| Challenge: | Existing benchmarks focus on product search tasks, but ignore potential risks. |
| Approach: | They propose a data generation pipeline that leverages webpage content and interactive elements to create diverse, functionality-grounded user queries. |
| Outcome: | The proposed framework assesses the performance and safety of web agents under dynamic, real-world e-commerce environments. |
FinRAG-12B: A Production-Validated Recipe for Grounded Question Answering in Banking (2026.acl-industry)
Copied to clipboard
| Challenge: | Large language models (LLMs) are rapidly being adopted across various domains, but adoption in the regulated banking industry is limited due to their tendency to hallucinate, exhibit over-agreeable behavior, and lack alignment with domain-specific knowledge and constraints. |
| Approach: | They propose a framework for training grounded domain-specific LLMs that optimizes answer quality, citation grounding, and calibrated refusal under real-world deployment constraints. |
| Outcome: | The proposed model outperforms GPT-4.1 on citation grounding and calibrated refusal under real-world deployment constraints. |
Towards Faithful and Robust LLM Specialists for Evidence-Based Question-Answering (2024.acl-long)
Copied to clipboard
| Challenge: | Evidence-Based QA has proved insufficiently faithful with Large Language Models . a typical application of LLMs is in Evidence-based Question Answering (QA). |
| Approach: | They propose a data generation pipeline with automated data quality filters to fine-tune LLMs for better source quality and answer attributability. |
| Outcome: | The proposed model can synthesize high-quality training and testing data at scale. |
HammerBench: Fine-Grained Function-Calling Evaluation in Real Mobile Assistant Scenarios (2025.findings-acl)
Copied to clipboard
Jun Wang, Jiamu Zhou, Xihuai Wang, Xiaoyun Mo, Haoyu Zhang, Qiqiang Lin, Jincheng Jincheng, Muning Wen, Weinan Zhang, Qiuying Peng, Jun Wang
| Challenge: | Evaluating the performance of LLMs in multi-turn interactions presents significant challenges due to the complexity and variability of user behavior. |
| Approach: | They propose a benchmark framework for assessing LLMs’ function-calling capabilities in multi-turn dialogues. |
| Outcome: | The proposed framework is based on a dataset derived from popular mobile apps and anonymized user logs. |
Falsesum: Generating Document-level NLI Examples for Recognizing Factual Inconsistency in Summarization (2022.naacl-main)
Copied to clipboard
| Challenge: | Neural abstractive summarization models generate factually inconsistent summaries . previous work has introduced the task of recognizing factual inconsistency as a downstream application of natural language inference (NLI). |
| Approach: | They propose a data generation pipeline that enables a task-oriented approach to detect factual inconsistencies in abstractive summarization models. |
| Outcome: | The proposed model improves the state-of-the-art performance across four benchmarks for recognizing factual inconsistency in generated summaries. |
FRUIT: Faithfully Reflecting Updated Information in Text (2022.naacl-main)
Copied to clipboard
| Challenge: | Existing work on generating edits grounded in external knowledge has focused on correcting grammar and reducing repetitive typing. |
| Approach: | They propose a novel task where the goal is to update an existing article given new evidence by using a dataset of 170K distantly supervised data produced from Wikipedia snapshots. |
| Outcome: | The proposed model can update Wikipedia articles faithfully with new capabilities and opens doors to many new applications. |
ParallelPARC: A Scalable Pipeline for Generating Natural-Language Analogies (2024.naacl-long)
Copied to clipboard
| Challenge: | Analogy-making is a central to human cognition, allowing us to abstract information and understand novel situations in terms of familiar ones. |
| Approach: | They propose a pipeline to generate paragraph-based analogies using large language models and large language distractors. |
| Outcome: | The proposed pipeline outperforms existing models in binary and multiple-choice settings and shows that humans outperformed the best models after a light supervision. |
Can LLMs Help You at Work? A Sandbox for Evaluating LLM Agents in Enterprise Environments (2025.emnlp-main)
Copied to clipboard
| Challenge: | Enterprise systems are crucial for enhancing productivity and strategic growth, but data is fragmented across multiple sources and access controls are complex. |
| Approach: | They propose a benchmark that simulates enterprise settings with 500 diverse tasks . they show that even the most capable models achieve only 41.8% task completion . |
| Outcome: | The proposed benchmark shows that even the most capable models achieve only 41.8% task completion. |
Improving Question Answering Model Robustness with Synthetic Adversarial Data Generation (2021.emnlp-main)
Copied to clipboard
| Challenge: | a new approach to generate adversarial data is needed to improve question answering models . crowdworkers can fool a model only 8.8% of the time, compared to 17.6% for a trained model without synthetic data. |
| Approach: | They develop a pipeline that generates questions and then filters or labels them to improve quality. |
| Outcome: | The proposed approach improves state-of-the-art on a human-written adversarial dataset by 3.7F1 and improves model generalisation on nine of the twelve MRQA datasets. |
Trajectory2Task: Training Robust Tool-Calling Agents with Synthesized Yet Verifiable Data for Complex User Intents (2026.acl-long)
Copied to clipboard
Ziyi Wang, Yuxuan Lu, Yimeng Zhang, Pei Chen, Ziwei Dong, Jing Huang, Jiri Gesi, Xianfeng Tang, Chen Luo, Qun Liu, Yisi Sang, Hanqing Lu, Manling Li, Jin Lai, Dakuo Wang
| Challenge: | Tool-calling agents are increasingly deployed in real-world customer-facing workflows . but most studies on tool-callers focus on idealized settings with general, fixed, and well-specified tasks. |
| Approach: | They propose a tool-calling agent-based data pipeline that converts trajectories into user-facing tasks with controlled intent adaptations. |
| Outcome: | The proposed pipeline can be used to study tool use under three scenarios. |